Who are we?¶
| Claudio | Bengt | Christophe | Per |
|---|---|---|---|
|
|
|
|
| Marcin | Ashfaq | Malin | |
|
|
|
Who are you?¶
Who are you?¶
Who are you?¶
Who are you?¶
Who are you?¶
Who are you?¶
Course website¶
[https://uppsala.instructure.com/courses/47910](https://uppsala.instructure.com/courses/47910)
- Learning outcomes
- Prerequisties/preparations
- Discussions
- Schedule / Modules
Feedback session on Friday¶
We are counting on your feedback!
- Does the schedule work the way it's laid out?
- What did you think of the lectures and labs?
- Too simple/hard?
- Write down one thing you liked and one you didn't at the end of each session, let us know at the end
Practical issues¶
- Try to keep your cameras on, but microphone muted
- Lectures with many breaks
- Take lots of small breaks also when working with the exercises
- We will try to stick to the schedule, but it's only preliminary until it's happened
- If you have any questions during the lecture, feel free to unmute and ask. If you don't want to ask in the Zoom meeting, write the question in the chat or on slack
So, this course is about Artificial Neural networks (Anns)¶
.... What is that?¶
AI vs ML vs ANN vs DL¶
(taken from https://medium.com/ai-in-plain-english/artificial-intelligence-vs-machine-learning-vs-deep-learning-whats-the-difference-dccce18efe7f)
AI -- Artificial Intelligence¶
- The science and engineering of creating intelligent machines
ML -- Machine learning¶
- Subset of AI
- Computer algorithms that learn/improve from experience
ANN -- Artificial Neural Networks¶
- Subset of ML
- Model of human neural networks
- Networks built up by artificial neurons
DL -- Deep Learning¶
- Subset of ANN (? -- depend on definition of ANN)
- ANN with more than one hidden layer
Where does ANN come from?¶
History¶
First wave of ANN (funding)¶
- 1943 First model of biological neruon (McCulloch & Pitts)
- 1949 Hebbian Learning rules for ANN, (Donald Hebb)
- 1951 First ANN (Minsky)
- 1958 The Perceptron -- linear classification
- 1969 Limitations of the perceptron (Minsky & Papert)
(taken from https://www.sciencedirect.com/topics/neuroscience/perceptron)
(taken from https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon)
History¶
Second wave of ANN (funding)¶
- (1975 Back propagation algorithm (Werbos))
- 1986 Back propagation rediscovered and sigmoid activation -- ANNs in vogue again
- 1985-87 ANN conferences, societies started (IEEE, INNS)
- Vanishing gradient problem
- SVM outcompetes ANNs, ANNs survives in protein structure prediction (e.g., CASP)
(taken from https://www.kdnuggets.com/2019/01/backpropagation-algorithm-demystified.html)
(taken from https://predictioncenter.org)
History¶
Third wave of ANN (funding)¶
- GPUs and parallelization
- "Deep Learning" buzzword and performing well in contests
- 1997 Recursive Neural Nets. LSTMs (Smidhuber & Hochreiter)
- 2000 ReLU (Hahnloser et al.?)
- 2011 CNNs
- 2020 AlphaFold2 souvereign at CASP14
- ANNs are everywhere!
(taken from https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)
Applications -- general¶
Written and spoke language recognition
(taken from Mouret, Jean-Baptiste & Doncieux, Stéphane. (2008). Evolutionary Intelligence
Risk predictions
(taken from https://spectrum.ieee.org/computing/embedded-systems/bringing-big-neural-networks-to-selfdriving-cars-smartphones-and-drones)
Applications -- general¶
Forecasting, weather, business
(taken from https://www.gjesm.net/article_23079.html)
Face recognition and generation
(taken from https://medium.com/syncedreview/gan-2-0-nvidias-hyperrealistic-face-generator-e3439d33ebaf)
Applications -- Biosciences¶
Protein structure prediction
